The present invention relates generally to database management systems having a primary database facility and a duplicate or backup database facility. More particularly, the present invention relates to system and method for purging database update image files after completion of associated transactions for a database replication system with multiple audit logs.
The present invention is an improvement on the Tandem xe2x80x9cremote data facilityxe2x80x9d (RDF) technology disclosed in U.S. Pat. No. 5,740,433, U.S. Pat. No. 5,745,753, U.S. Pat. No. 5,794,252, U.S. Pat. No. 5,799,322, U.S. Pat. No. 5,799,323, U.S. Pat. No. 5,835,915, and U.S. Pat. No. 5,884,328, all of which are hereby incorporated by reference as background information.
Prior art RDF technology underwent a number of changes over time to increase the peak number of transactions per second that can be performed on the primary system and replicated on the backup system. In a current adaptation, RDF technology uses multiple audit logs, or audit trails, to keep track of database updates and backouts of RDF-protected database volumes. With the utilization of multiple audit logs, a large increase in the rate at which transactions performed on the primary system can be replicated on the backup system. The utilization of multiple audit logs violates basic assumptions of the prior art systems, requiring both redesign of prior art mechanisms and some completely new mechanisms, to ensure that the backup system maintains xe2x80x9csoft synchronizationxe2x80x9d with the primary system during normal operation, and to also ensure that the backup system can be brought to an entirely consistent internal state whenever the backup system needs to perform a takeover operation and be used as the primary system.
In particular, one of the new mechanisms that need to be revamped is the file purging mechanism. In prior art RDF systems, where there is only a single audit trail, the mechanisms for determining whether an image trail file may be purged (permanently deleted) is rather straight-forward. The above-mentioned co-pending United States patent application entitled xe2x80x9cSYSTEM AND METHOD FOR PURGING DATABASE UPDATE IMAGE FILES AFTER COMPLETION OF ASSOCIATED TRANSACTIONSxe2x80x9d describes such a technique. The utilization of the multiple audit logs creates another level of complexity and suggests the need for a new file purge mechanism for purging unneeded image trail files.
In summary, the present invention provides for an efficient method for determining whether an image file contains audit records associated with transactions that have not yet committed or aborted such that image trail files can be safely deleted without adverse consequence. The present invention is preferably practiced in a distributed computer database system having a local computer system and a remote computer system. The local computer system has a local database stored on local memory media, application programs that modify the local database, and a transaction manager that stores audit records in multiple local audit trails reflecting those application program modifications to the local database. The transaction manager stores in a particular one of the local audit trails transaction state records indicating the transaction states of the transactions making those database modifications. The valid transaction states of a transaction can be committed, aborted, active, aborting or prepared. The particular local audit trail is referred to as a MAT (master audit trail). The other local audit trails are referred to as AuxATs (auxiliary audit trails). The transaction manager also stores in the MAT a type of records known as Auxiliary Pointer Records, which indicate the range of audit records in the AuxATs that were flushed to disks since the last Auxiliary Pointer Record.
The remote computer system, remotely located from the local computer system, has a backup database stored on remote memory media associated with the remote computer system.
A remote duplicate data facility (RDF) is partially located in the local computer system and partially in the remote computer for maintaining virtual synchronization of the backup database with the local database. The RDF includes multiple Extractor processes that execute on the local computer system, and multiple Receiver processes and multiple Updater processes that execute on the remote computer system.
A Master Extractor process extracts audit records from the MAT, and each of the Auxiliary Extractor processes extracts auxiliary audit records from one of the AuxATs. The Extractor processes, when extracting audit records from the MAT and the AuxATs, insert an Audit Trail Position (ATPosn) value in each audit record. The Extractor processes then transmit the extracted audit records to the remote computer system.
The Receiver processes receive the extracted audit records from the Extractor processes and distribute the extracted audit records to one or more image trails in the remote computer system. Each Receiver process is also responsible of storing the ATPosn of the last audit record it received. The Receiver processes are each mapped to one of the Extractor processes. A Master Receiver process receives audit records from the Master Extractor, and each Auxiliary Receiver processes receive audit records from one of the Auxiliary Extractor processes. The audit records include audit update and audit backout records indicating database updates and database backouts generated by transactions executing on the local computer system. Control-type audit records, which only appear in the MAT, are distributed to a Master Image Trail (MIT). Data-type audit records of the MAT are distributed to MAT-based Secondary Image Trails (SITs). Audit records of the AuxATs are distributed to AuxAT-based SITs. Note that data-type audit records of the MAT or the AuxATs may be distributed to more than one SITs. In one embodiment of the invention, data-type audit records and transaction state records contain Transaction IDs each consisting of a system (node) number, a CPU number and a transaction sequence number. In some embodiments, a transaction ID may consist of a transaction sequence number and a system number or a CPU number and/or some other information for uniquely identifying a transaction.
The Master Receiver uses a counter to maintain a count of TMP Control Point Records it received. The counter is known as the TMP Control Point Count. When the Master Receiver receives a TMP Control Point Record, it durably stores the current SysTxList record, initializes a new SysTxList record, increments the TMP Control Point Count and stores the TMP Control Point Count in the new SysTxList record. The Master Receiver updates the SysTxList record whenever it receives a Transaction State Record (transtate record), an Auxiliary Pointer Record, a data-type record or any audit record having one or more Transaction ID(s). When the Master Receiver receives another TMP Control Point Record, the SysTxList record is considered completed and is durably stored. The Master Receiver then initializes a new SysTxList record.
A SysTxList record contains information indicating the range of active transactions that were processed by the local computer system during a particular TMP Control Time Frame. Specifically, a SysTxList record contains a TMP Control Point Count that is associated with the TMP Control Time Frame. A SysTxList record also contains, for each CPU in each Node of the local computer system, the Lowest Transaction Sequence Number (Lo_Tx_Seq_Num) and the Highest Transaction Sequence Number (Hi_Tx_Seq_Num) of the transactions that were processed during that particular TMP Control Time Frame. The SysTxList also contains, for each Auxiliary Audit Trail, the lowest Low-Water-Mark and the highest High-Water-Mark of the Auxiliary Pointer Records of that particular TMP Control Time Frame. SysTxList records are stored in SysTxList Lookup Files.
Image Trails are stored in Image Trail Files that have sequentially numbered file names and a fixed file size (configurable for each system). When the Master Receiver creates a new Image Trail File, it writes the current TMP Control Point Count (i.e., the TMP Control Point Count that has been written to the current SysTxList record) in the file""s header. When an Auxiliary Receiver opens a new Image Trail File, it writes the Audit Trail Position of the audit record that the Auxiliary Receiver most recently received from the corresponding Auxiliary Extractor in the file""s header. The information contained in the headers of the files will be used for determining whether the files can be purged.
For each SIT there is an Updater that applies to a backup database volume the database updates and backouts indicated by the audit update and audit backout records in the SIT. The audit update and audit backout records are applied to the backup database volume in same order that they are stored in the image trail, without regard to whether corresponding transactions in the primary system committed or aborted. An Updater reads the Image Trail Files one file at a time and keeps track of the TMP Control Point Count or High-Water-Mark position it extracted from the current file""s header. Periodically, a MAT-based Updater sends the extracted TMP Control Point Count to the Purger, and an AuxAT-based Updater sends the extracted High-Water-Mark to the Purger.
The Purger periodically builds a HWM-to-TMP_Ctrl_Pt_Cnt Table and Composite SysTxList for determining whether an Image Trail File contains audit records that may be needed by the Updater processes for performing an Updater Undo Process. The Composite SysTxList contains, for each CPU in each NODE of the local computer system, the lowest transaction sequence number and the highest transaction sequence number for transactions that are considered xe2x80x9cactivexe2x80x9d by the Updaters. The HWM-to-TMP_Ctrl_Pt_Cnt Table contains, for each TMP_Ctrl_Pt_Cnt and for each Auxiliary Audit Trail, a Low Low-Water-Mark and a High High-Water-Mark of Auxiliary Pointer Records of that TMP Control Time Frame. After the Purger constructed the HWM-to-TMP_Ctrl_Pt_Cnt Table and Composite SysTxList, the Purger accesses the Image Trail Files, and for each Image Trail File the Purger retrieves a corresponding SysTxList and compares it against the Composite SysTxList to determine whether any earlier Image Trail Files can be purged.